Models

Most of the structure of likelihood is same as that in hierarhical models with binary latent variable except that Bernoulli distributions now turn into Multinomial distributions. Models are based on [model 2] \(P(A, X, Z) = P( A | Z) P(Z | X) P(X)\).

\[X_{i} \overset{i.i.d}{\sim} Multinomial(u_{1}, u_{2}, 1 - u_{1} - u_{2} ), i = 1,... , n\]

Note that \(X\) should not be interpreted as a nominal, categorical variable since Euclidean distance was used to measure the distance of \(X\).

\[\begin{align} Z_{i} | X_{i} & \overset{i.i.d}{\sim} Multinomial( 1 ; \pi_{1}(X_{i}), \pi_{2}(X_{i}), \pi_{3}(X_{i}) ) = Multinomial(1; \pi_{k} = (1/3 + \omega) I(X_{i} = k ) + (2/3 - \omega ) I(X_{i} \neq k) /2 , k = 1,2,3 ) \\ & \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinomial(1; \color{red}{\frac{1}{3} + \omega}, \frac{2/3 - \omega}{2} , \frac{2/3 - \omega}{2} ) & X_{i} = 1 \\ Multinomial(1; \frac{2/3 - \omega}{2}, \color{red}{\frac{1}{3} + \omega} , \frac{2/3 - \omega}{2} ) & X_{i} = 2 \\ Multinomial(1; \frac{2/3 - \omega}{2}, \frac{2/3 - \omega}{2}, \color{red}{\frac{1}{3} + \omega} ) & X_{i} = 3 \end{array} \right. \end{align}\]

\[\begin{align} P_{\phi}(A = a | Z = z) & = \prod\limits_{k} P_{\phi}(A_{ij} = a_{ij} | Z = z) \times \prod\limits_{k < l} P_{\phi} (Y_{kl} = y_{kl} | Z = z) \\ & = \prod\limits_{Z_{i} = Z_{j} = 1}^{K} {p_{k}}^{a_{ij}}(1-p_{k})^{a_{ij}} \prod\limits_{Z_{i} \neq Z_{j}} {q_{kl}}^{a_{ij}}(1 - q_{kl})^{a_{ij}} \end{align}\]

\[P_{\theta}(A, X, Z) = P_{\phi}(A | Z) P_{\omega}(Z | X) P_{u}(X)\]

[sim] three.1

\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]

\[Z_{i} | X_{i} = Z_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1,... , n.\]

\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{0.5} & 0.1 & 0.1 \\ \hline 0.1 & \color{red}{0.5} & 0.1 \\ \hline 0.1 & 0.1 & \color{red}{0.5} \end{array} \right]\]

u1 u2 w p1 p2 p3 q12 q13 q12
0.33 0.33 0 0.5 0.5 0.5 0.1 0.1 0.1

t=1 t=5 t=20
global test 0.08 0.05 0.06
local optimal 0.11 0.14 0.13


[sim] three.2

\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]

\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]

\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{0.5} & 0.1 & 0.1 \\ \hline 0.1 & \color{red}{0.5} & 0.1 \\ \hline 0.1 & 0.1 & \color{red}{0.5} \end{array} \right]\]

u1 u2 w p1 p2 p3 q12 q13 q23
0.33 0.33 0.17 0.5 0.5 0.5 0.1 0.1 0.1

t=1 t=5 t=20
global test 0.82 0.87 0.84
local optimal 0.93 0.96 0.92


[sim] three.3

\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]

\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]

\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} 0.3 & 0.3 & 0.3 \\ \hline 0.3 & 0.3 & 0.3 \\ \hline 0.3 & 0.3 & 0.3 \end{array} \right]\]

u1 u2 w p1 p2 p3 q12 q13 q23
0.33 0.33 0.17 0.3 0.3 0.3 0.3 0.3 0.3

t=1 t=5 t=20
global test 0.06 0.06 0.11
local optimal 0.09 0.16 0.14


[sim] three.4

\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]

\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]

\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{0.5} & 0.1 & 0.1 \\ \hline 0.1 & \color{red}{0.5} & 0.1 \\ \hline 0.1 & 0.1 & 0.1 \end{array} \right]\]

u1 u2 w p1 p2 p3 q12 q13 q23
0.33 0.33 0.17 0.5 0.5 0.1 0.1 0.1 0.1

t=1 t=5 t=20
global test 0.89 0.73 0.75
local optimal 0.93 0.87 0.89


[sim] three.5

\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]

\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]

\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{0.5} & 0.1 & 0.1 \\ \hline 0.1 & 0.1 & 0.1 \\ \hline 0.1 & 0.1 & 0.1 \end{array} \right]\]

u1 u2 w p1 p2 p3 q12 q13 q23
0.33 0.33 0.17 0.5 0.1 0.1 0.1 0.1 0.1

t=1 t=5 t=20
global test 0.57 0.66 0.62
local optimal 0.80 0.75 0.71


[sim] three.6

\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]

\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]

\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{0.5} & 0.1 & 0.1 \\ \hline 0.1 & 0.1 & 0.1 \\ \hline 0.1 & 0.1 & \color{red}{0.5} \end{array} \right]\]

u1 u2 w p1 p2 p3 q12 q13 q23
0.33 0.33 0.17 0.5 0.1 0.5 0.1 0.1 0.1

t=1 t=5 t=20
global test 0.82 0.88 0.89
local optimal 0.98 0.96 0.95

Summary

Summary 1

\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]

\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]

\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{p} & q & q \\ \hline q & \color{red}{p} & q \\ \hline q & q & \color{red}{p} \end{array} \right]\]

Power heatmaps

  • Global Power

  • Optimal Power

Optimal power

Discrepancy between local and global scale

Diffusion Process



Summary 2

\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]

\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]

\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{p} & r & r \\ \hline r & \color{red}{p} & r \\ \hline r & r & r \end{array} \right]\]

Power heatmaps

  • Global Power

  • Optimal Power

Optimal power

Discrepancy between local and global scale

Diffusion Process



Summary 3

\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]

\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]

\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{p} & q & \color{blue}{r} \\ \hline q & \color{red}{p} & q \\ \hline \color{blue}{r} & q & \color{red}{p} \end{array} \right]\]

Since block 1 (\(Z_{i} = 1\)) and block 3 (\(Z_{i} = 3\)) are most different, if \(q < r,\) local scale is more likely to be better than the global scale. Thus I on purpose consider the case where \(q < r.\)


r = q + 0.1

Power heatmaps

  • Global Power

  • Optimal Power

Optimal power

Discrepancy between local and global scale

Diffusion Process


r = q + 0.2

Power heatmaps

  • Global Power

  • Optimal Power

Optimal power

Discrepancy between local and global scale

Diffusion Process